4? 



TRANSMITTAL OF API 



IEF (Small Entity) 



Docket tio. 
CYB-07003/03 



In Re Application Of: Paul et al 



AUB 1 6 2D04 & I 



Serial No. 


Filing Da^*A0gS$ 


* Examiner 


Group Art Unit 


09/896,150 


June 29, 2001 


Scott Jones 


3713 



Invention: REAL-TIME HEAD TRACKING FOR COMPUTER GAMES AND OTHER APPLICATIONS 



TO THE COMMISSIONER FOR PATENTS: 

Transmitted herewith in triplicate is the Appeal Brief in this application, with respect to the Notice of Appeal filed on: 
June 14, 2004 

Applicant is a small entity under 37 CFR 1 .9 and 1 .27. 

A verified statement of small entity status under 37 CFR 1 .27: 

□ is enclosed. 

□ has already been filed in this application. 

The fee for filing this Appeal Brief is: $165.00 
IS A check in the amount of the fee is enclosed. 

IS The Director has already been authorized to charge fees in this application to a Deposit Account. 

IS The Director is hereby authorized to charge any fees which may be required, or credit any 
overpayment to Deposit Account No. 07-1180 



Dated: August 12, 2004 



/ Signature 

John G. Poja 

Reg. No. 3^,424 

Gifford, Krass, Groh et al 

280 N. Old Woodward Ave., Suite 400 

Birmingham, MI 48009 

Tel. 734/913-9300 

Fax 734/913-6007 




1 certify that this document and fee is being deposited 
on August 12, 2004 with the U.S. Postal Service as 
first class mail under 37 C.F.R. 1.8 and is addressed to the 
Commissioner for Patents, P.O. Box 1450, Alexandria, VA 
22313-1^50. jj i 




Signatt\tp of Person Mailing Correspondence 


cc: 


Sheryl L. Hammer 


Typed or Printed Name of Person Mailing Correspondence 



P30SMALL/R EV03 




THE UNITED STATES PATENT AND TRADEMARK OFFICE 
BOARD OF PATENT APPEALS AND INTERFERENCES 




CO 



% Appellant: 

cn 

CO 
10 

§ Serial No.: 



O 
GO 



< Filing Date: 



George V. Paul et al. 

09/896,150 

June 29, 2001 



Group Art Unit: 3713 
Examiner: Scott E. Jones 



i 
o 



2 For: REAL-TIME HEAD TRACKING FOR COMPUTER GAMES AND OTHER 
APPLICATIONS 



< 
x 
o 
z 

DC 

CO 

O 
O 



APPELLANTS' BRIEF UNDER 37 CFR S1.192 



^ Mail Stop Appeal Brief 
yj Commissioner for Patents 
gPOBox 1450 

% Alexandria, VA 22313-1450 
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gDear Sir: 
§ 

_l 1. 

o 



Real Party in Interest 

o The real parties and interest in this case are inventors George Paul, Glenn Beach, Charles Cohen, 
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^ and Charles Jacobus, Applicants and Appellants. 
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Related Appeals and Interferences 

There are no appeals or interferences which will directly affect or be directly affected by or have 



8 a bearing on the Board's decision in the pending appeal. 
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*III. 



Status of Claims 

rr 

& The present application was filed with 13 claims. Claim 11 was canceled in August 2003. 

i 

§ Claims 1-10 and 12-13 remain pending and are under appeal. 
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IV. Status of Amendments Filed Subsequent 
Final Rejection 

An after-final amendment is submitted herewith to correct the priority claim. 
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~ V, Concise Summary of the Invention 



g This invention resides in a real-time computer vision system capable of tracking moving objects 

CO 

in 

g in a scene. Unlike current search and locate algorithms, the subject algorithm uses a target location 

CO 

z technique which does not involve search (Specification, page 4, lines 2-4). The system tracks objects 



i based on the color, motion and shape of the object in the image. The tracking algorithm uses a unique 



color matching technique which uses minimal computation. This color matching function is used to 



(3 compute three measures of the target's probable location based on the target color, shape and motion. It 
| then computes the most probable location of the target using a weighting technique (Specification, page 
1 4, lines 4-9). These techniques make the invention very computationally efficient also makes it robust 

Lii 

k to noise, occlusion and rapid motion of the target (Specification, page 4, lines 9-10). 

LU 

g The imaging hardware of the real-time object tracking system includes a color camera, a frame 

o grabber, and a personal computer. The software includes low-level image grabbing software and the 
| 

0 tracking algorithm (Specification, page 4, lines 1 1-13). Once the application is running, a graphical user 

o 

^ interface displays the live image from the color camera on the computer screen. The operator can then 

_i 

° use the mouse to click on the hand in the image to select a target for tracking (Specification, page 4, 

g 

cm lines 13-15). The system will then keep track of the moving target in the scene in real-time. As such, 

d 

*the system finds utility in a wide range of applications, including head tracking for game control 

CO 

1 (Specification, page 4, lines 15-22). 
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o VI. Concise Statement of Issues Presented 
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For Review 



1. Areclaims 1-10 and 12-13 unpatentable under 35 U.S. C. § 103(a) over U.S. Patent No. 
§6,075,557 to Holliman et al. in view of the Birchfield technical paper? 
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£ VII. Grouping of Claims for Each Ground of 
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Rejection Which Appellants Contend 

None. 
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| VIII. Argument 

Si 

* Claims 1-10, 12 and 13, wherein claims 2-10, 12 and 13 stand or fall claim 1. 

O) 3 3 2 

CO 

g Claims 1 stands rejected under 35 U.S.C. §103(a) as being unpatentable over Holliman et al., 

o 

CO 

2 U.S. Patent No. 6,075,557, in view of Birchfield. Among other elements, claim 1 includes the 

< 

CD 

x limitation of "providing a weighted average of color to compute the location of the user's head based 

| upon color alone." The Examiner concedes that Holliman et al. are silent with respect to this aspect, but 
< 

o argues that Birchfield address this deficiency citing page 2, column 2 to page 3, column of the 



| Birchfield paper. The Examiner is incorrect. 

m 

§ Birchfield teaches a method for object tracking that combines the output of two different 

LU 

& modules: one that matches the intensity gradients along the object's boundary, and one that matches a 
z color histogram of the object's interior. The latter, color module considers the color of a human head 
o to be complex and being at the very least bimodal due to the usually contrasting colors of the skin and 
§ hair. Birchfield consequently uses a color histogram in its method for target tracking because of the 

8 

5 ability of the color histogram to inclusively capture complex, multi -modal patterns of color of the kind. 
9 The Birchfield reference goes on to teach the computation of a histogram intersection between a 

o 

8 model histogram and an image histogram at each hypothesized location of the user's head. The paper 
* goes on to explain that the power of the histogram intersection results from the MIN( ) function, which 
| matches no more image pixels of a certain color that are present in the model histogram. Thus, for 
o example, the measure of accuracy for object tracking is more satisfied within a region containing both 
o facial and hair color than a region containing all facial color. 



o It is well-settled that if a proposed combination of prior- art references would change the 

< 

uj principle of operation of the prior art invention being modified, then the teachings of the references are 
Inot sufficient to render the claims prima facie obvious. In re Ratti, 270 F.2d 810, 123 USPQ 349 
x (CCPA 1959). Such is the case at bar. Birchfield teaches using the MIN( ) function to determine the 

DC 

^.histogram intersection between a model histogram and an image histogram for object tracking based on 

c color alone. Appellants claim the use of a weighted average of color to compute the location of a user's 
o 

ghead based upon color alone. 

LL 
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o Furthermore, in rejecting claims under 35 U.S.C. §103, the Examiner must provide a reason 

why one having ordinary skill in the pertinent art would have been led to combine the cited references to 
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~ arrive at Appellants' claimed invention. There must be something in the prior art that suggests the 
* proposed combination, other than the hindsight gained from knowledge that the inventor choose to 

CO 
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g combine these particular things in this particular way. Uniroyal Inc. v. Rudkin-Wilev Corp. , 837 F.2d 

o 

| 1044, 1051, 5 USPQ2d 1434, 1438 (Fed. Cir. 1988). The Examiner is also required to make specific 

< 

i findings on a suggestion to combine prior-art references. In Re Dembeczak , 175 F.3d 994, 1000-01, 50 

|uSPQ2d 1614, 1617-19 (Fed. Cir. 1999). 
< 

| The Examiner has not met these requisite standards. As for motivation to combine, the 



qc Examiner argues that the combination "provides for a robust head tracking system that is accurate 

m 

§ enough to accurately control the camera's pan, tilt and zoom for long periods of time in order to keep 

LU 

te the user's head in the field of view at the desired size." However, these are considerations not found in 

LU 

z Appellants' claims, or in the prior art for that matter. For example, Appellants can find no evidence in 

g the primary reference of Holliman et al. regarding panning, tilting or zooming. As such there can be no 

| 

o motivation to improve that which does not exist. Appellants do not claim, nor does the prior art of 

o 

£ record recite a need for "accurate control ... for long periods of time in order to keep the user's head in 

_i 

° the field of view at the desired size." 
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* Conclusion 

g In conclusion, for the arguments of record and the reasons set forth above, all pending claims of 

othe subject application continue to be in condition for allowance and Appellants seek the Board's 
o concurrence at this time. 
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APPENDIX A 



CLAIMS ON APPEAL 



< LA method controlling a computer game, comprising the steps of: 
i 

y imaging a sequence of scenes including the head of a user of the computer; 

< comparing visual characteristics from a portion of a scene to a center of said portion of a scene 
?to determine movement of the user's head within the scene wherein at least one of the visual 
m characteristics is color; 

o 
o 

* providing a weighted average of color to compute the location of the user's head based upon 

So 

w color alone; and 

z 

I controlling the game in accordance with the movements. 

Q 

QC 
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g 2. The method of claim 1, wherein the visual characteristics include shape or location. 

<: 



| 3 . The method of claim 2, wherein the visual characteristics include a combination of static 

00 
CM 

^and dynamic characteristics. 



§ 4. The method of claim 3, further including the step of modeling of the dynamic 



^characteristics to yield an estimate of head position. 
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| 5. The method of claim 1 , further including the step of initiating head tracking through a 

UJ 

I graphical user interface. 
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| 6. The method of claim 5, wherein the graphical user interface provides a bounding box 

^displayed in a screen to assist in targeting the user's head. 

QC 

t 7. The method of claim 2, further comprising the step of enabling a match in color despite 

o 



the differences arising from lighting and shadows. 
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& 8. The method of claim 2, further comprising the step of enabling a match in color within a 

S threshold of hue. 

CO 

m 

Ci 

o 

§ 

z 9. The method of claim 1 , wherein the step of comparing the visual characteristics includes 

o 



x a comparison of pixels from scene to scene. 
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0 10. The method of claim 1, further including the step of determining if the user's head was 

1 moved outside of the scene. 
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^ 12. The method of claim 1, further including the step of segmenting a region defined by a 

g predetermined closeness of color as an estimate of target shape. 
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g 13. The method of claim 1 , further including the step of continuing to track the user' s head 

o 

2 when moving in front of or behind a similarly colored object in the scene. 
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